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C\) ■ Abstract 



We introduce oscillatory analogues of fractional Brownian motion, sub-fractional Brownian 
motion and other related long range dependent Gaussian processes, we discuss their properties, 
and we show how they arise from particle systems with or without branching and with different 
types of initial conditions, where the individual particle motion is the so-called c-random walk 
on a hierarchical group. The oscillations are caused by the discrete and ultrametric structure 
of the hierarchical group, and they become slower as time tends to infinity and faster as time 
approaches zero. We also give other results to provide an overall picture of the behavior of this 
kind of systems, emphasizing the new phenomena that are caused by the ultrametric structure 
as compared with results for analogous models on Euclidean space. 
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1 Introduction 
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^ ■ The main objectives of this paper are to introduce oscillatory analogues of fractional Brownian 

motion, sub-fractional Brownian motion and other related long range dependent Gaussian 
processes, and study their properties, and to show how those processes arise from systems 
of hierarchical random walks. In this Introduction we give some relevant background and put 
the paper in context. 

It is well known that fractional Brownian motion (fBm) with Hurst parameter H G (0, 1/2)U 
(1/2, 1), whose covariance function is 

has an intrinsic mathematical interest and a wide variety of areas of application that are 
often mentioned in papers on the subject (see [ST] for basic background and [DOTj for some 
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applications). The main properties of fBm are that it is a continuous centered Gaussian process 
which is self-similar, with stationary increments and long range dependence, and it does not have 
the Markov property and is not a semimartingale (the case H = 1/2 corresponds to ordinary 
Brownian motion). A closely related process is sub-fractional Brownian motion (sfBm) with 
parameter H £ (0, 1/2) U (1/2, 1), whose covariance function is 

s 2H + t 2H -\[(s + t) 2H + \s- t\ 2H ], s, t > 0. (1.2) 

It shares the properties of fBm except increment stationarity, and it has a faster decay of 
long range dependence (the case H = 1/2 also corresponds to ordinary Brownian motion). 
This process was analyzed in [BGTOJ and in |BGTlj . where it was shown that it arises from 
occupation time fluctuations of a branching system of particles moving according to a symmetric 
a-stable Levy process in M. d , and H is restricted to (1/2, 1). A similar result in |BGTlj for a 
system without branching leads to fBm with H restricted to (1/2,3/4]. SfBm also appeared 
independently at the same time in |DZj in a different context as the even part of fBm (defined 
on the whole real line). In |BGT9] it was found, again for a system of a-stable processes, 
that sfBm also arises in a natural and intrinsic way (in fact, more natural than fBm) not 
necessarily related with branching. A different type of particle picture interpretation for fBm 
and sfBm with values of H in the whole interval (0,1), also based on a system of a-stable 
processes, was given in |BTj . where the role of time is played by a spatial parameter. In a 
series of papers jBGTOl IBGTll IBGT21 IBGT31 IBGT41 IBGT51 IBGT6| IBGT71 IBGT81 IBGT9] , 
dealing with various models within a large class, we have studied occupation time fluctuations 
of a-stable particle systems that lead in particular to fBm, sfBm and other related processes. 
Some of those papers will be referred to below for comparisons or for some technical points. 
Other authors have recently investigated some analogous models where those processes and 
similar ones also arise [BB| IBZ| ILX| IM1} IM2} IM3} ISw] , further properties of sfBm and related 
stochastic calculus have been dis cussed [BBl EH EH E3 ILYPWl Ei El ED E H ISCl ISYll 
ISY21 [SZJ HU [T2l HU El [T5l lYSllYSH] . a strong approximation has been devised [GGL] . and 
an application has been proposed |LLYj . Another process that possesses the main properties 
of fBm except increment stationarity, and has also been studied by several authors recently, 
is bi-fractional Brownian motion, which was introduced in |HVJ . According to some authors, 
processes that do not have stationary increments may be more useful for modelling in some 
applications (e.g., |BMG] in financial markets). 

The oscillatory analogues of fBm and sfBm are (centered Gaussian) oscillatory fractional 
Brownian motion (ofBm) £ H , with parameter H £ (1/2, 1) and covariance function 

Egg = \{s 2H K + t 2H h t -\s- t\ 2H h ]s _ tl ), s,t>0, (1.3) 

and oscillatory sub-fractional Brownian motion (osfBm) ( H , with parameter H £ (1/2, 1) and 
covariance function 

Egg = s 2H h s + t 2H h t -^[(s + t) 2H h s+t + \s-t\ 2H h ls „ tl ], s,t>0, (1.4) 

where (h t ) t>0 is a periodic function in logarithmic scale, i.e., h t = h at , t > 0, a being a constant, 
< a < 1. This function oscillates between two positive values, slower as t oo, faster as 
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t — > 0. A detailed description of these processes and other related ones is given in the next 
section. If the function h were constant, ofBm and osfBm would reduce to fBm and sffim, 
respectively. The function h that arises from the models cannot be constant, hence ofBm and 
osfBm may not be considered as extensions of fBm and sfBm, but they can be regarded as 
oscillatory analogues of them. 

Another process is oscillatory negative sub-fractional Brownian motion (onsfBm), which has 
a covariance of the same form as that of osfBm, but with H G (1,3/2) and a slightly different 
oscillatory and negative function instead of h. This is an oscillatory analogue of the negative 
sub-fractional Brownian motion (nsfBm) that was found in [BGT7] with a high density limit 
and H G (1,5/4). In [BTJ there is also a different type of particle picture interpretation for 
nsfBm with the full range H G (1,2). 

It could be that the oscillatory analogues of fBm, sfBm and nsfBm turn out to be useful 
for some applications due to the additional "volatility" introduced by the oscillations of their 
covariances and their paths, specially for small times, while keeping most of the other properties 
of the respective non-oscillatory processes. 

The oscillatory analogues of fBm, sfBm and nsfBm come about when instead of systems of 
a-stable processes we consider systems of (a class of) hierarchical random walks. 

The state space for hierarchical random walks is the hierarchical group of order M (integer 
> 1), also called hierarchical lattice in physics, which is defined by 

= {x = (x 1 ,x 2 ,...) : Xi G {0,1,..., M- 1}, EiXi < oo}, (1.5) 

with addition componentwise mod M. It is a countable Abelian group which is the direct sum 
of a countable number of copies of the cyclic group of order M. A translation invariant metric 
| ■ | on Q M , called hierarchical distance, is defined by 

\x-y\ = l ° if X = V > (16) 

[ max{« :Xi^yi}iix^y, 

and satisfies the strong (non- Archimedean) triangle inequality 

\x — y\ < max{|x — z\, \z — y\}, x,y,z G VLm- 

Therefore (fijy, | • |) is an ultrametric space. Background on ultrametric spaces can be found 
e.g. in [Sc] . The main difference with spaces of Euclidean type is that in an ultrametric space 
two balls are either disjoint or one is contained in the other. As a consequence, contrary to 
the Euclidean case, the only way to "go out" of a ball of radius R is to make a single jump 
of size larger than R, and it is not possible by a sequence of small steps. (OaTi I'D can be 
represented as the set of leaves at the top of an infinite regular tree where each inner node 
at each level j > 1 from the top has one predecessor at level j + 1 and M successors at level 
j — 1. The distance between two individuals (leaves) at the top is the depth in the tree to their 
most recent common node. Hierarchical structures of this type arise in the physical, biological, 
social and computer sciences due to the multiscale organization of many natural objects. For 
example, Sawyer and Felsenstein |SF] have used a model like this where the distance represents 
the degree of genetic relatedness between individuals in a population that are grouped at several 
different levels of organization. Hierarchical structures have also been used in statistical physics, 
fundamentally in the work of Dyson [DJ on ferromagnetic models (see also |CE^ EI]), as they 
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provide "caricatures" of Euclidean lattices which make it possible to carry out renormalization 
group analysis in a rigorous way. Some papers that use hierarchical structures in connection 
with population models, branching systems, interacting diffusions, self-avoiding random walks, 
contact processes, percolation, and search algorithms are \AS\ iBTj IGDGl IDGoll IDGo2t IDGWl], 
IDGW31 IDGTI IDU21 IDUZl MM £2 KU K] . Hierarchical random walks play a key role in some 
of the models in those papers, as do simple symmetric random walks in models on Euclidean 
lattices. 

A hierarchical random walk on VLm is defined by a probability distribution ( 

Tj)j>i on 

{1,2,...}. The walk first chooses a distance j with probability rj, and then jumps to a point 
uniformly among all those that are at distance j from its previous position. We assume that 
there exist arbitrarily large j such that 77 > 0, for otherwise the walk would never leave a 
bounded set, due to the abovementioned fact on jumping out of a ball. A particular hierarchi- 
cal random walk that will be the most relevant in this paper is the c-random walk, where c is 
a constant, < c < M, and 

A continuous time c-random walk with jump rate 1 has transition probability for going from 
to y in time t > given by 

00 f n 

Pt (o,y) = e- t T-p {n \a,y), (1-8) 

n=0 

where p^ n \0,y) is the transition probability from to y in n steps for the discrete time walk. 
In what follows, c-random walk will always refer to the continuous time process. 

The c-random walk "mimics" (or is a "caricature" of) a symmetric a-stable process on M. d in 
a sense explained below. Some properties of a-stable processes and c random walks (and other 
Levy processes on additive Abelian groups) can be described in a unified way by means of the 
degree 7, which is the supremum over all the o > — 1 such that the operator power G a+1 of the 
potential operator G of the underlying motion process is finite. For 7 > 0, the degree also gives 
information on how fast the process escapes to infinity. The degree is useful for comparison of 
results on particle systems involving a-stable process and corresponding ones with c-random 
walks. The development of hierarchical random walks has been outlined in |DG W2] . and their 
degree and several of their properties, in particular recurrence/transience, have been studied in 
[DGW4J (see also a summary in the Appendix of |BGT10j ). 

For symmetric a-stable process on M. d , 7 is given by 

d 

7 = --l, (1.9) 

a 

and for c-random walk on Qm (either in discrete or continuous time), 7 is given by 

log c 

7 = MMA)- (L10) 

For a-stable process 7 is restricted to the interval [—1/2, 00), and for c-random walk 7 can 
take any value in (—1, 00). In this sense c-random walks are a richer class of processes. This is 
the reason why in the occupation time fluctuation limit referred to above for a-stable process 



4 



without branching the parameter H (which is expressed in terms of 7 in the same way for 
both processes) in (jl.ip is restricted to the interval (1/2,3/4], whereas for the c-random walk 
it can take any value in (1/2, 1), as we shall see in Theorem 2.2 and Remark 2.3(a). A similar 
thing happens with the nsfBm (for a-stable process), which has H G (1, 5/4) [BGT7J, and the 
onsffim (for c-random walk), which has H € (1,3/2) (Theorem 2.6). 

The sense in which c-random walks are considered to "mimic" a-stable processes is that 
they have the same recurrence/transience behaviors for equal values of their degrees, and that 
they have the same spatial asymptotic decay of powers of their potential operators in terms of 
the "Euclidean radial distance" (see |DGW4j . Remark 3.2.2(b)). But there are also differences, 
for example, the distance of the c-random walk from a given point behaves differently than 
the a-stable Bessel process (see [DGW4], Remark 3.5.7). Branching c-random walks can be 
considered similarly as branching a-stable processes, with a given branching rate. 

In |DGW1] . occupation time fluctuation limits for branching particle systems were studied 
(with one and two branching levels, and also without branching), where the individual particle 
motion was either symmetric a-stable process on M, d or c-random walk on Qm- I n both cases 
the results referred to the occupation time fluctuation at a final time t as t — > 00. A much 
more comprehensive asymptotics of the occupation time of the systems consists in accelerating 
the time t by a scaling parameter T and studying the limit of the fluctuation process for 
t G [0, 00) as T — > 00, as this reveals in detail the time behavior properties of the limit process. 
That was the objective of our papers in the case of a-stable process on M. d (we considered 
the models without branching and with only one branching level, but also with other types 
of initial conditions and other differences with respect to |DGWlj ). The typical situation is 
that there are two different limiting regimes, corresponding to "low" and "high" dimensions 
d, and a "critical" dimension that represents the phase transition between the two regimes. 
Those dimensions depend on whether or not branching is present. For "low" dimensions the 
limit process has long range time dependence (fBm in the case without branching, and sfBm 
with branching) and a uniform spatial structure (Lebesgue measure), for "high" dimensions 
it has independent increments in time and a non-trivial spatial structure (e.g., a non-trivial 
5'(]R d )-valued Wiener process), and at the "critical" dimension the fluctuations have a larger 
order of magnitude (with a logarithmic factor), the spatial structure is uniform and the time 
behavior is Brownian motion. Phase transition phenomena are well known in several systems 
of interest in statistical physics. 

Concerning the spatial structure of the limits, the three behaviors recalled above were 
already found in |DGW1] for both a-stable process and c-random walk, and they were given 
in terms of 7 in a unified way. Namely, the regimes for the non-branching model in the "low" , 
"critical" and "high" cases are —1 < 7 < 0,7 = 0,7 > (i.e., 1 = d < a,d = a,d > 
a;c<l,c=l,c>l, respectively), and for the branching model (1 level branching), they are 
0<7<1,7 = 1,7>1 (i.e, a < d < 2a, d = 2a, d > 2a; 1 < c < a/M, c = a/M, c > \/M, 
respectively). 

The natural question that remained was the possibility of extending results of [DGWlJ on 
c-random walks by adding the time structure, as described above, analogously as it was done 
in our previous papers for a-stable processes. This is the problem solved in the present paper. 
Clearly, the same regimes occur with the phase transition between long range dependence in 
the "low" case and independent increments in the "high" case. The main novelty now is the 
appearance of the oscillatory analogues of fBm and sfBm in the cases of long range dependence, 
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namely, ofBm for — 1 < 7 < (without branching), and osfBm for < 7 < 1 (with branching), 
in both cases with Poisson initial condition, the latter also arising from a system without 
branching but with a different type of initial condition, and also onsfBm (with branching and 
high density, — 1 < 7 < 0) (Theorems 2.2, 2.4 and 2.6). Generally, for the models with a-stable 
processes or c-random walks, in the "low" cases the most interesting are the time properties 
of the limit processes, and in the "high" cases the spatial properties are the most relevant. 
Nevertheless, in all cases there are also significant differences between the two models. In 
the "low" cases the discrete and ultrametric structure of the state space Qm causes the time 
oscillations of the limit processes, and the limits exist only for special sequences T n — > 00. In 
the "high" cases, at a fixed time the spatial structure of the limit exhibits fractional behavior 
for a-stable processes (see [BTj ). while for c-random walks it is Markovian of diffusion type 
(see Subsection 2.3.11). 

In Section 2 we begin by describing the systems of hierarchical random walks (interpreted 
as particle motions) with different types of initial conditions that have effects on the limit 
processes, and we give the definitions of ofBm, osfBm, onsfBm, and other related processes 
that will appear. We then formulate convergence theorems for non-branching and branch- 
ing systems, and discuss the results. After that, we study properties of the obtained limit 
processes, including certain forms of self-similarity, long range dependence, non-Markov and 
non-semimartingale properties for ofBm and osfBm, and these two processes are represented 
in terms of a sequence of independent Brownian motions (Proposition 2.14). On the other 
hand, onsfBm is a semimartingale (by Proposition 2.16), analogously as nsfBm [BGT5j. We 
also give a result on quadratic variation (Proposition 12. 18[) that exhibits an oscillatory (in a 
sense) character of paths of ofBm and osfBm. Finally we point out the Markov property of the 
spatial structure in the "high" cases. 

In Section 3 we give the proofs. Since our main interest is identifying the limit processes, 
we prove functional convergence (i.e., tightness in addition to convergence of finite dimensional 
distributions) only for some of the main limits, focussing on the cases where the oscillatory 
limits occur. A new way of proving convergence that was introduced in |BGT9] (different 
from the method in our previous papers) is again useful here and allows to give reasonably 
short proofs of the limit theorems; some of the ideas did not appear in the published version 
[BGT9J, but they can be found in the complete version in arXiv. The methods of proof have 
some analogies to those used in the Euclidean case with stable processes, but the ultrametric 
structure of Qm involves different types of calculations. 

In Section 4 several additional questions are mentioned in a series of comments. They 
include results that complete the overall picture, some conjectures and some additional things 
that could be done with other models, but we do not go into them because they would not be 
significantly new and they would extend the length of the paper considerably. 

The following notation is used in the paper. 
Bb- space of real functions on Qm with bounded support; 
B' b : dual of B\,\ 
(-, •): duality on B' b x B^] 

%: semigroup associated with c-random walk, i.e., 




(1.11) 
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where pt(x, y) = p t (0,y — x) is given by ( 11. 8ft ; 
G: potential operator (for 7 > 0), i.e., G<p(x) = f °° %<p(x)dt; 
weak convergence of finite dimensional distributions; 

weak convergence of distributions in the space of continuous functions C([0, r]) for each 
r > 0. 

Generic constants are written C, Cj, with possible dependencies in parenthesis or in subscripts. 



2 Results 



2.1 Description of the problem 

Given an integer M > 1, let Qm be the hierarchical group defined in the Introduction (see 
( ll.5p .( TTTB|) ). On Qm we consider a system of continuous time random walks defined in the 
following way. At time t = 0, at each x G Qm there are 9 X particles, where {0 x }xem are 
independent copies of a random variable 9 with values in {0, 1,2,.. .}, such that E8 3 < 00. A 
typical case is that 6 is a Poisson random variable. Such a system will be called Poisson. Each 
particle evolves according to a (continuous time) c-random walk independently of the others, 
with the same c (see (11.7jl ). Sometimes we assume additionaly that the particles branch at 
rate V independently, and the branching is critical and binary (0 or 2 particles with equal 
probabilities). The evolution of the system is described by the empirical process (N t ) t > , where 

N t (A) = number of particles in the set A C Qm at timet. 

For T > 0, we define the occupation time fluctuation process 

1 f Tt 

X T (t) = — / (N r - EN r )dr } t > 0, (2.1) 
Ft Jo 

where Ft is a suitable deterministic norming. We are interested in the asymptotics of the law 
of Xt as T —7- 00. Unlike the system of a-stable motions in R d , in the most interesting cases a 
limit of X T exists only for special sequences T n — > 00. 



2.2 Convergence results 

We begin by defining the processes that will appear in the limits; their existence will follow from 
the corresponding convergence results and their properties will be given in the next subsection. 
Denote 

c M 2 — c 

fl ~M' M(M-l)' ( } 

(our b is ba in the notation of [DGW1] ). 

Oscillatory fractional Brownian motion (ofBm) with parameter H G (1/2, 1) is a centered 
Gaussian process £ H = (£ t H )i>o with covariance given by (II. 3p (hence it clearly has stationary 
increments), where 

h t = ^{ba j t)- 2H {e- baH -1 + baH), t > 0. (2.3) 
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Convergence of this series is easy to check, and 

< d(a, H) <h t < C 2 (a, H) < oo, (2.4) 

for all t > 0, with 

Cx(a,H) = C 2 (a,H) = a 1 - 2H C 1 (a } H) } (2.5) 

(see [DGW3] . (4.2.8) and Proposition 3.1.1). We additionally define h = 0. Note that 

h at = K t> 0, (2.6) 

which justifies the qualifications "oscillatory" and "periodic in logarithmic scale". Observe that 
ht is infinite for H G (0, 1/2]. The constant h in (12.21) appears in the limit, but it is irrelevant 
for the existence and properties of the process defined by (jl.3p and (j2.3P ; one may take any 
b > 0. 

Oscillatory sub-fractional Brownian motion (osfBm) with parameter H G (1/2,1) is a 
centered Gaussian process ( H = (Ct ? )t>o with covariance given by (jl.4p and h t defined by (12. 3p . 

Oscillatory negative sub-fractional Brownian motion (onsfBm) with parameter H G (1, 3/2) 
is centered Gaussian process ( H = (C/^)t>o with covariance of the form (jl.4p with h replaced by 

h t = J2(baH)~ 2H (e~ baH - 1 + baPt - -{ba j t) 2 \ , t > 0. (2.7) 

Using e~ x — 1 + x — ^ = — Jq Jq Jq e~ v dvdzdy, x > 0, it is easy to check that is well defined 
and | h t \ satisfies (12. 4p with 

CAa, H) = r(4-2iJ) = / ^ / 2 8 n 

u ' ; 2H(2H- 1)(2H- 2)(a 2H - 3 - 1)' v ' 7 u ' ' y 1 

h obviously satisfies (I2.6P and h t < 0. Note that putting formally h t = —1 we obtain the 
negative sub- fractional Brownian motion considered in |BGT7j . 

Remark 2.1. It is useful to observe that the covariances of both osfBm and onsfBm can be 
written in a unified form: 

rt ps puAv 

/ / / ^2e~ ba3{u+v ~ 2r \ba j f- 2H drdudv. (2.9) 

JO J0 Jo .gg 

After computing the integrals, (12. 9 p takes the form 

Y^^-Bj), ,1,. Hj > 0. 

where the series ^ ■ Aj, ^ ■ Bj converge if H G (1/2, 1) (hence it can be written as ^ — . Sj, 
yielding (II. 4ft with /i of the form (12. 3ft ) and diverge if if G (1, 3/2). In the latter case it is nec- 
essary to take ^2j(Aj — Cj), J2j(Bj — Cj) for appropriate Cj, hence the different forms of h 

and h. For example, the existence of osfBm and onsfBm follows immediately from (12.91) . The 
expression (12.91) is finite and positive definite also for H = 1, and the corresponding Gaussian 
process is an analogue of the process obtained in (2.14) of |BGT7j . and it is discussed in [BGT5J 
(formulas (3.5), (3.6)). However, in this case (12.91) cannot be given a form like (11. 4ft . 
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Another process that appears as an ingredient of the occupation time fluctuation limit is a 
centered Gaussian process i] H with covariance 

EVsVt = (s + t) 2H h s+t - t 2H h t - s 2H h s , s,t>0, (2.10) 

where h is given by ( 12 .3p and H G (1/2, 1). This is an oscillatory analogue of a process studied 
in jENl ES E23 and, independently, in |BGT9j . 

We now give the convergence results. In what follows, K is a constant whose value can be 
calculated explicitely in each case. This is our oscillatory analogue of a process introduced in 
[LN] for H G (0, 1/2), and extended in (EEHET] for H G (1/2, 1). 

Theorem 2.2. Consider a system of c-random walks without branching and c < 1 (j < 0, see 
f ll.lOP ), and a general initial condition determined by 9 (see Subsection \2.1\) . Let Xt be defined 
by (J23} with 

Ft = T (1 " 7)/2 . (2.11) 

Then, for 

T n = a~ n } n = l,2,..., (2.12) 

and any <p G Bb, 

(X Tn ,<p) ^K(V2E6( H + VV^9 V H ) J2 <P(x), (2.13) 

with ( H and i] H independent, and H — (1 — j)/2 G (1/2, 1) (i.e., ( H is osfBm). 

Remark 2.3. (a) If the distribution of 9 is Poisson or, more generally, E9 = V&rO, then the 
limit process in (I2.13P is Ki(<p)£ . If 9 is deterministic, then this limit is K 2 ((p)C H ■ 

(b) This result is an analogue of Theorem 2.2 in |BGT9j . It exhibits the relevant role of osfBm 
for the model. 

(c) From the proof it will be seen that instead of T n of the form ( 12.121) one can take any sequence 
(T n ) n such that T n a kn has a positive limit for some k = 1,2, . . .. Then there exists a unique 
mo G Z and K G [1, 1/a) such that T n a kn a m ° — > n, and the limit of Xt u has the form (12.13!) . 
where in the definition of h (see (12. 3p ) b is replaced by Kb. 

(d) We have formulated the convergence results for a fixed test function ip G Bb only, but as 
VIm is countable and discrete, it is easily seen that the (weak) convergences take place also in 
the space of continuous processes with values in the dual B' b . 

Remarks (c) and (d) apply also to the next two theorems. 

We now pass to the branching case. 

Theorem 2.4. Consider a Poisson system of c-random walks with binary branching at rate V . 
We assume that 1 < c < a/M (0 < 7 < 1). Then, with 

F T = T( 2 -^/ 2 , (2.14) 

T n of the form (12.121) and any ip & Bb, 

(X Tn ,p)^K( H J2 ¥>(*), (2-15) 

where ( H is osfBm with H = (2 - j)/2 G (1/2, 1). 
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Remark 2.5. This theorem is an analogue of Theorem 2.2 for a-stable motions in [ BGTlj . 
The latter result was extended to general initial conditions of homogeneous type with the same 
limit process in |BGT9j . However this extension was not immediate. We are sure that ( 12 . 1 5[) 
also holds for any initial condition described in Subsection 2.1, but we have not made detailed 
calculations. 

In order to obtain onsfBm it is necessary to add high density to the previous model. 

Theorem 2.6. Consider a Poisson system of c-random walks with binary branching at rate V. 
We assume that c < 1 (7 < 0) and the Poisson intensity of the initial distribution is Ht > 
such that 

lim T 7 # T = 00. (2.16) 

T->oo 

Then, with 

Ft = T (2 ~ 7)/2 i4 /2 , (2-17) 

T n of the form (12.121) and any <p e 

(X Tn ,<p)^K( H J2 (2-18) 

where ( H is onsfBm with H = (2 - j)/2 e (1, 3/2). 

Remark 2.7. This theorem is an analogue of Theorem 2.2(a) of [BGT7j (see also Remark 
2.3(a) and Proposition 2.5 therein). In |BGT7] a convergence result was also given for a = d, 
which in our case corresponds to c = 1 (7 = 0, see (11.91) . (11.101) ). For such c one can obtain 
(F2~T8|) as well with ( H whose covariance is of the form (I2.9P with H = 1 (see Remark 2.1). 

We now pass to the case of "large" 7. 

Proposition 2.8. Consider a Poisson system of c-random walks without branching with 
c> 1 (7 > 0). Let F T = VT. Then 

X T =p- X as T — > 00, 

where X is a centered Gaussian process with values in the dual B' b with covariance 

E(X(s),<p)(X(t),ip) = 2E6(sAt)(G<p,^), (2.19) 

where G is the potential operator of the particle motion process. 

Remark 2.9. (a) We formulate explicitly the result for the Poisson model without branching 
only, since in other cases the limit processes are similar. It seems clear that analogously as for 
the a-stable motions in M d (see [BGT9j . complete version), replacing Poisson random measure 
by an arbitrary initial distribution described in Subsection 2.1 does not change the limit. In 
the branching case with 7 > 1 (c > VM), a result of jDGWlj (Theorem 2.2.2), and |BGT2j 
and |BGT9] (complete version, a-stable motions) indicate that, again with Ft = VT, the limit 
process is the sum of two independent centered Gaussian processes, one of the form (I2.19p . and 
the other with covariance E0V{s/\t){G?y,i)). In the critical cases, i.e., 7 = (no branching) 
and 7 = 1 (branching), with F T = y/T log T, the limits are of the form Kfi(3, where f3 is the 
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standard one dimensional Brownian motion, and \x is the counting measure on Qm- As these 
results do not seem particularly interesting, we dispense with the calculations. 

(b) So we see that for "large" 7, analogously as for a-stable motions, the limit processes have 
independent stationary increments. Moreover, the form of the covariances is expressed in terms 
of the potential operator G in the same way. However, an interesting feature of the process 
with covariance ( I2.19P is that its spatial structure is completely different from the corresponding 
limit for a-stable motions. We will see this in Theorem 12.211 and Remark 2.22. 

(c) Observe that for "large" 7 no oscillations occur, therefore to obtain convergence there is no 
need to take special sequences T n (see ( 12.121) ). 



2.3 Properties of the limit processes 
2.3.1 Different forms of the covariances 

Proposition 2.10. (a) Let h be given by ( 12. 3 p and H G (1/2, 1), then 



t^ht = r S ^M 2 a(2u)du, (2.20) 



v? 



where 



(b) For of Bra £ we have 



a(u) = - > j— '- -. 2.21 



E£(j* = { - - co < su ^ 1 - cos (f)) + sin ^) si ^ tu K(u)du. (2.22) 
Jo u 

(c) For osfBm and onsfBm ( H ,H G (1/2, 1) U (1,3/2), we have 



EgC? = I (1 " C ° SM) 1 1 " CQSM V h^. (2.23) 



Remark 2.11. (a) Formulas (I2T22P and (|2T23|) for H G (1/2, 1) follow immediately from (|2~20|) . 
but the fact that (I2.23P also holds for onsfBm (H G (1,3/2)) is not so immediate since the 
covariance is expressed in terms of h given by (12. 7p . and not h. The covariance (II. 3p with the 
representation (I2.20I) - (I2.21I) (extending t 2H ht by symmetry to t < 0) is of the form related to 
"helical arcs" studied in |NSj (see also \AL\ IKah] ). 

(b) From this proposition it is clear that ofBm, osfBm and onsfBm can be defined for all t G K, 
if in (11.31) and (jl.4p we put \t\, \s\, \t + s\. Then the process 

is an osfBm (this relationship for fBm and sfBm was observed in jBOTOj ). 

The next simple proposition exhibits another aspect of analogies between the structures 
of oscillatory processes and their non-oscillatory counterparts. The corresponding formulas for 
the non-oscillatory processes turned out to be useful in [BTJ . 
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Proposition 2.12. (a) For ofBm £ we have 

EZf&= [ l M {x)l m {y)Y\M^e-^^dxdy. (2.24) 

(b) For osfBm ( H ,H G (1/2, 1), we have 

ECf Cf = \f (lM - l[-.,o])(aO(l[o,t] - l H ,o])(l/) ^M^eH'^'drdy. (2.25) 

(c) For onsfBm ( h ,H G (1,3/2), we /iave 

^Cf Cf = ~ E / ( J M - l[- S ,o])(x)(l [M - l Hj o])(y)(6^') 2 - 2H e-l a: -^ 6af fcdy. (2.26) 

Remark 2.13. (a) (I2.26P holds also for osfBm, but for onfsBm the sum cannot be put under 
the integral. 

(b) For fractional Brownian motion and sub-fractional Brownian motion with H G (1/2,1) 
there are analogues of (I2.24p and f!2.25j) with kernel C\x — y\ 2H ~ 2 in |BT] (Corollary 2.12 and 
Remark 2.13(a)). 

2.3.2 Representations in terms of a sequence of independent Brownian motions 

Proposition 2.14. Let {^}jez be independent standard Brownian motions and {vj}jei be an 
i.i.d. family of standard normal random variables, independent of {^}j e z- 
(a) Let H G (1/2, 1). The process 

J2(ba j y H (l - e- baH )v 3 + V2j2(ba j ) 1/2 ~ H [ (1 - e'^^d^, t > 0, (2.27) 



is an ofBm. 
(b) The process 

Y^{bo?) 1/2 - H [ (1 - e- hai{t - r) )dPi, t>0, (2.28) 
is an osfBm if H G (1/2, 1) and an onsfBm if H G (1,3/2). 

Remark 2.15. (a) the first series in (I2.27P is the process r/ H defined by (I2.10p . and (I2.27P 
corresponds to the representation of ofBm resulting from ( I2.13P for E9 = Var# (see Remark 
2.3(a)). 

(b) It is not clear how to represent ofBm and osfBm in terms of just one stochastic integral. 
For fBm there are representations with one stochastic integral, see |Nu] and references therein; 
for sffim, see jBGTOj . 
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2.3.3 Integral representation of onsfBm 

Negative sub-fractional Brownian motion with parameter H can be represented as C J Q * g u du 
where g is the odd part in the decomposition of fBm with parameter H — 1 considered in [DZJ 
(see |BGT5j . Theorem 3.1). A similar fact is true for onsfBm with an oscillatory analogue of g. 

Proposition 2.16. OnsfBm (H e (1,3/2)) has a representation 

Cf = / Qu^du, t>0, (2.29) 
Jo 

where g u is a continuous centered Gaussian process with covariance 
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Eg u s g u t = 2 [(« + ty u h u s+t -\s- trh u {s _ A \ , U G (0, 1/2), (2.30) 
and 

h v t = ^{baHy 2U {\ - e- baH ), t > 0. (2.31) 

The integrand process in the analogue of f !2.29j) for the nsfBm has covariance of the form 
( gjjQD with h = 1. 

2.3.4 Bounds for second moments of increments and Holder continuity 
Proposition 2.17. Let d H denote either one of the processes £ H or ( H , then 

Cl \t - sf H ^ 2 < E(#* - 7?f ) 2 <C 2 \t- s\W A2 , (2.32) 

where C\ and Ci are positive constants that depend on £ H or ( H , respectively; in particular for 
osfBm ^ H these constants are given by (12. 5p . 

Hence ofBm and osfBm have versions whose paths are locally Holder continuous with any 
exponent smaller than H. On the other hand, Proposition 2.16 implies that the paths of onsfBm 
are continuously differentiable. 

2.3.5 Semimartingale/non-semimartingale property 

From f |2.32j) and Lemma 2.1 in |BGT] it follows that ofBm and osfBm are not semimartingales. 
It can be also shown that the centered Gaussian process with covariance (12 .9p for H = 1 is not 
semimartingale either. On the other hand, onsfBm is obviously a semimartingale by (12. 29 p . 
The process r] H defined by (I2.10p is also a semimartingale, since from ( I2.27P and Remark 12.151 
(a) it follows that it can be written in a form similar to (I2.29p . 
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2.3.6 Weighted quadratic variation 

The result of this subsection a first attempt to clarify how the oscillatory character of the co- 
variance function is reflected in the behavior of paths. From (I2.32p it follows that the quadratic 
variation of the paths of ofBm and osfBm is on each interval [0, r\. We now study the rate of 
convergence to 0. An analogous problem for fBm and sfBm was solved in \Q\\ |N| ITT] . 

Proposition 2.18. Fix T > 0, H G (1/2, 1) and let $ be either ofBm or osfBm 
(a) For e > 0, let 

V e {&) = e" 2// 



/ (A+e-Afdt. (2.33) 
Jo 



Then 

lim — V e (ti) = T (2.34) 
in L 2 , where h e is given by (12.31) . Moreover, if (e n )n=i,2,... is such that 

oo 

4 < 00 f° r some < <J < (4 - 4ff) A 1, (2.35) 

71=1 

then for e = e n the convergence in f )2.34p as n — )■ oo is almost sure as well, 
(b) The same assertion holds for 

VT/e\ 

UM = e^ 2H (#ke - (2-36) 
fc=i 

(where |_-J stands for integer part). 

Remark 2.19. (a) For a sequence e n = na n , k G [1,1/a), (12.351) is obviously satisfied, and 
then we have 

lim V £ M = lim U £ M = Th K 

n— ><x> n— >oo 

in L 2 and almost surely. This result exhibits the oscillatory (in a sense) character of the paths, 
(b) In |T1] the p-variation of sfBm is studied. It seems that similar methods would allow to 
investigate ofBm and osfBm. 



2.3.7 Non-Markov 

None of the covariances of the oscillatory processes has the triangular property [Kal] (Proposi- 
tion 13.7), hence those processes are not Markov. 



2.3.8 Self-similarity 

£ H and ( H are not self-similar, but we have 

)t>o 

for A = a\ j G Z, by (12.61) and (jl.3p . Moreover, for general A, if we write it as A = a^K, 
K G [1,1/a), j G Z, then 

[$\t)t>o — {A Q )t>o, 
where ^ n,H is ofBm with b replaced by bn (see (12.31) ). The same is true for Q H . 
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2.3.9 Long range dependence 

Proposition 2.20. Let 0<u<v<s<t, then 
(a) 

< C^(u,v,s,t) <lhninir 2 - 2H E(C ~^M +T -e+r) 

T— >00 

< limsupr 2 -^E(e - Cf)fe+. - C+r) < C?(u,v, s,t) < oo, 

T — ^OO 

for He (1/2,1), 



< ^(n, W , S ,t)<liminfr^E(C-C)Cf + .-Cf +r ) 

T— >-00 

< limsupr 3 - 2 "^ - Cf)(Ct+ T - < Cf f) < oo 

for H G (1/2,1) U (1,3/2), where the Cf (u,v, s,t) are positive constants for each u,v,s,t. 

The orders of decay of the covariances of increments for the oscillatory processes are the 
same as the orders for their non-oscillatory counterparts (see [BGTCH IBGT5] , the result is well 
known for fBm). 

2.3.10 Asymptotics for the boundary values of H 

Both fBm and sfBm are the standard Brownian motion for H = 1/2, and there is a continuous 
dependence on H. We have already mentioned that offim and osfBm are not defined for 
H = 1/2. Nevertheless, from (pHL and we have that 



\/l - a 2H -^ H j^K(3 as H \ 1/2, 

where (3 is standard Brownian motion. The same holds for osfBm ( H . 

Similarly, for onsfflm, from (EZD, (EED and (O) it follows that if H / 3/2, then Va 2 ^" 3 - 1C H 
converges in the sense of finite dimensional distributions to nsfBm with H = 3/2 (see [B GT5j 
IBGT7] ). 

2.3.11 Spatial structure for "large" 7 

We restrict our considerations here to a Poisson non-branching system with 7 > (c > 1). In 
the remaining cases the results will be either identical or similar (see Remark 2.9(a)). With no 
loss of generality we assume the Poisson parameter E9 = 1. 

Let B n denote the ball in Qm with center and radius n = 0, 1, . . .. Denote 



M — 1 c , 

D = , 2.37 

Mb c-1' K ' 



where b is given by (12. 2p . 
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Theorem 2.21. Let X be the limit process given by Proposition \2.8\ and denote 



t = {X(l),l Bt ), t = 0,l,.... (2.38) 
Then d is a centered Gaussian process with covariance 

/M\ sVt 

Ed s § t = 2DM sAt ( — J . (2.39) 

Remark 2.22. (a) From (12.391) it follows that the process *& and its continuous time inter- 
polation resulting from (I2.39P are Markov ( [Kal] . Proposition 13.7). This is in sharp contrast 
with the a-stable case. Recall that 7 > corresponds to 1 = d > a for the Poisson system of 
a-stable motions in M. without branching. If X is the occupation time fluctuation limit process 
(see |BGT2] or |DGWlj ). then the counterpart of (I2.38P is (X (1) , l[^ t ,t]) , an d this process is 
the odd part of the fBm in the sense of [DZ] . i.e., a centered Gaussian process with covariance 
of the form 

K (\ t + S \™ -\t- s \ 2H ), H= 1 -^. 

This process is not Markov. In the a-stable case it makes sense to consider also the processes 
(X(l), l[o,t]) an d (X(l),l[ 0jt ] — l[_t i0 ]), which are fBm and sfBm, respectively [BT] (Corollary 
2.12), see also [LXJ. So the results of [BGT2] and |BT] show that in this model, as 7 changes 
from "small" to "large" values, there is a phase transition in the sense that a fractional tem- 
poral structure and a uniform spatial structure changes into a simple temporal structure and 
a fractional spatial structure. In the hierarchical case we have a passage from oscillatory frac- 
tional temporal structure and uniform spatial structure to a simple temporal structure and a 
Markov (non-fractional) spatial structure. 

(b) It is easy to see that the process •& can be obtained as a limit of ((X T (l), lB t ))t=o,i,...- 

(c) The hierarchical structure of the state space Qm is the reason that the logarithmic scale is 
more natural. So, we consider the process d defined by 

tf t = tfio gl/at , t >0. (2.40) 

This is a centered Gaussian process, and by (12.391) its covariance is given by 

Efisfit = 2D(s A t) 1+7 (s V t). (2.41) 

By ( I2.40p . d t is defined for t = a~ n ,n = 0, 1,2, . . ., but a Gaussian process with covariance 
(I2.4ip is well defined for any t > 0. The interpolation is also denoted by The usefulness of 
this type of logaritmic scaling and the relevance of the sequence a~ n (see (I2.12p ) are also seen 
in [BGTlOj . 

We now collect some properties of the process defined by (I2.40p . which follow immediately 
from (jMED . 

Proposition 2.23. The centered Gaussian process $ with covariance (I2.4ip . with 7 > 0, has 
the following properties: 
(i) it is Markov, 
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(ii) it is self-similar with index 2 + 7, 
(Hi) it can be represented as 

d t = ^/2D^t [ u^ /2 d(3 u , 
Jo 

where P is a standard Brownian motion, 

(iv) it is of diffusion type, satisfying the stochastic differential equation 

d# t = -# t dt + ^/2D^t^ +1 ^ 2 d(3 t . 

t 

3 Proofs 

We start by recalling some facts on c-random walks. First observe that (11.101) and (12.21) imply 

h = a7+1 - (31 > 

This permits to write the formula for the transition probability, given, e.g., in (3.1.3) of [DGWlj . 
in the following form 

p t (x, x + y) = p t (0, y) = (l {0} (y) - l)a^ +1 ) e - 6>M < + £ a^e~ baH (3.2) 

j=\y\ 

(see (12.21) for our definition of b). This and an easy estimate, 

< Ci < J2( bajt y +le ~ baH <C 2 , 1> -1, t > 0, (3.3) 

yield 
Denote 



Pt(o,y) < Pt(o,o) < C 3 (i Ar^). 


(3.4) 


G t ip(x) = / T s (f(x)ds, ip G B b , 
Jo 


(3.5) 


( C(</?)r 7 if -1< 7 < 


(3.6) 


GMx) < I C(ip)\ogt if 7 = 


(3.7) 


lc(<p) if 7>0. 


(3.8) 



Proof of Theorem 12.21 

Recall that 9 X is the initial number of particles at x G Qm, and {6 x }xen M are independent 
copies of 6. Denote 

p k = P(e = k), fc = 0,l,... (3.9) 

Let iV^ be the empirical process of the system starting from 9 X particles at x. By assumption, 
the N( x \ x G &m > are independent and 

JV = N(x) - ( 3 - 10 ) 
17 



This representation permits to use the central limit theorem in the proof of convergence of 
finite dimensional distributions. 

For simplicity, we will prove the theorem for ip = l{o}- It will be clear that the same 
argument can be carried out in the general case. 

Let {z x ' j } xen M ,j=i,2,... be independent c-random walks. Then 

(Nt\v) = J2v(x + z^). (3.11) 

First we show convergence of covariances. Fix t > s > 0. We have 

Ccn{(N<?\<p),(N? , \<p)) = I-I, (3.12) 



where 



/ = E{NS>\<p)(Ni m \<p) 

oo k oo k 

k=l i,j=l k=l j=l 

= (E9 2 -E8)p s (0,x)pt(0,x) + E8p s (0,x)pt- S (0,0), (3.13) 



and 



I = E(Nj?\ p)E(Ni x \cp) = (E9) 2 Pt (0, x) Ps (0, x). (3.14) 
This, (I3.10P and the Chapman-Kolmogorov equation imply 

Cov((iV s , ip), (N t , ip)) = E9p t _ s (0, 0) + (Var 9 - E9)p t+S (0, 0). 

Hence, by (ED) , 

Cov((X Tn {s),<p), (X Tn {t),<p)) = E9A n + (Vartf - E9)B n , (3.15) 



where 



t„ Jo jo 

From (EEED, and (Q it follows that 



A n = ^- f [ p Tnlu - vl (0,Q)dudv, (3.16) 

^T n JO JO 

B n = ^f [ p Tn{u+v) (0,0)dudv. (3.17) 

JO JO 



M Jo Jo ._ n 
Using ( 13. 3 p and 7 < we have 



j=0 



M — 1 f s f l 

\imA n = —— ya i(7+1 »e^ IH Wt><oo. (3.18) 

n ^°° M Jo Jo j-j 



j 
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Analogously, 

limE n = / Y ^ +l) e~ ba3[u+v) dudv <oo. (3.19) 

m y Jo ^ 

Computing the integrals in (13.181) and (13.191) . by (13. 15j) we obtain 

lim Cov((X Tn (s)^), (X Tn (t),<p)) = K 2 (2E9ECC t H + Ve,v9E V frjf) 1 (3.20) 

n— >oo 

with # = (1 - 7 )/2 and K 2 = (M — l)/(Mb 1+ ^) (see ([HD, (£D and (I27TU1) ). 

Observe that existence of 7/ H and £ H for H e (1/2,1) follows from (I3.20[) . Namely, 
taking # deterministic we obtain positive defmiteness of (11.41) . Next, taking E9 small and 
keeping Var 6 = 1 we see that (12. lOj) is also positive definite. Finally, existence of £ H is seen by 
taking Ed = Var 6. 

Now we pass to the proof of convergence of finite dimensional distributions. 

For fixed a 1; . . . , a m G K and ti, . . . , t m > 0, we write the sum Y^!k=i a k(X Tn {t k ) , if) with 
the help of (13. 1Q|) and we apply the Lyapunov criterion. It is not hard to see, using E6 3 < oo, 
that it suffices to prove that 



( I l ^( x + z u 1 ) du ] asn^oo (3.21) 



1 n 



(see the end of Subsection 3.2 in [BGT9], or Subsection 3.3 in the complete version) 
We have, using (13. 5p and (II. lip . 



6 

St = f3 / / / %( l P%-r( l pTt- s ip))(x)dtdsdr 

^ T „^c, Jo Jr J s 

V] / ip{x)G T {^G T ^){x)dr 

„^n . JO 



6 

7 1 3(l- 7 )/2 



T (l-3 7 )/2 

(recall that 90 = l{o})- By (13. 6[) we obtain 

C 



° G T (ipG T ip)(0) (3.22) 



y(l+ 7 )/2' 



hence ( 13.211) follows. 

It remains to prove tightness of (X<r n ,y?) in C([0, r]) for any r > 0. From ( I3.15j) ( 13. 17j) it 
follows that, for s <t, 

E((X Tn (t),<p) - (X Tn {s)^)f = E6A' Tn + (Vartf - E6)B' Tn , 

where 

rp2 rt rt rp2 rt rt 

A' T = -~2 / pr\u-v\(0,0)dudv, B' T = — j / / p T ( M+ „)(0,0)d«dt>. 

-*T J s J s "t J s J s 
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We have 

ft— s rv fT(t-s) 



Pt — S PV PI [t—s ) 

A' T < 2T 1+7 / / p Tr (0, 0)drdv < 2T 7 (t - s) p r (0, 0)dr < C(t - s) 1 " 7 

Jo Jo Jo 

by ( 13. 6ft . As t H- pt(0, 0) is decreasing, then 

pt—S pt—s 

B' T < T 1+7 / / p Tr (0, 0)drdv < C{t - S y-\ 
Jo Jo 

Hence tightness follows. This completes the proof of the theorem. □ 
Proof of Theorem 12.41 

Without loss of generality we may assume that the Poisson parameter of the law of 9 is 1. We 
follow the same scheme as in the previous theorem. Again we restrict to the case ip = l{o}- 

First we show convergence of covariances. A standard argument for branching Poisson 
system yields 

(puAv 
<p(x)T\ v - u \<p{x) + V ip(x)T u+v -2r^{x)d 

where, for our tp, by ( 12. ip and ( I2.14p . after an obvious substitution we have 

Cov((X T (s),cp)(X T (t),cp)) = I(T) + 1(T), (3.23) 

where 2 ^ 

I(T) = ^J o j o Pt\v-u\(0, 0)dudv, (3.24) 

rp3 pt PS puAv 

E{T) = V— J J J p T{u+v „ 2r) {0,0)drdudv. (3.25) 

Assuming that s <t, 

pt pt pTt 

I(T) < 2T 7 / / p Tu (0, 0)dudv = 2tT 7 " 1 / p u (0, 0)du -> (3.26) 
Jo Jo Jo 

as T — > oo, by ( 13. 8p . since 7 < 1. 
From ( I3.2p . for T n = a~ n we have 

"t ps puAv 00 



m — 1 r r s r 

E[Tn) = v Mb^J 11 J2( bai y +le ~ bal(u+v ~ 2r)drdudv 

j=~n 

-+ / / / Y.^ ] v +le ~ ba3(u+v ~ 2r)drdudv 

*/ J */ j 
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by ( 12. 9p . with H = (2 — r y)/2. Then (I3.23p - fl3.27p imply the desired convergence of the covari- 
ance function of (Xt, <p). 

Thanks to (I3.10P we can prove convergence of finite dimensional distributions using the 
central limit theorem (Lyapunov criterion). Analogously as in the proof of Theorem 2.2 this 
reduces to showing that 



S T := -^3 E ( I ' ^ dv ) ~* as T ~* 00 ' 



(3.28) 



where N x is the empirical process of a branching system starting from a single particle at x. 
Define 



v g (x,t) = 1 - Eexp f-q J (N%,(p)ds^ . 



A standard argument (conditioning on the first branching and applying the Feynman-Kac 
formula, see e.g. [TJLMJ or [B GTlj ) shows that 

V q (x,t) = f %-a { q<p(l - V q (-,S)) ~ —«„(-, S) J [x)ds. 



Hence, by a similar reasoning as in (3.45)-(3.47) of |BGT2] (see also the proof of Theorem 2.4 
in [BGT9j . complete version) we obtain 



h(T) + I 2 {T) + h{T) + J 4 (T), (3.29) 



T xen M 



where 



h(T) = -^3 ^2 J Tt-u J Tu- Ul (^P J T Ul - U2 vdu^j du^j (x)du, 

h(T) = ^ ^2 J T T ~u((pJ %- Ul (^J T Ul -u 2 pdu^j du x j (x)du, 

h{T) = ^ J T T -u (J^J T u - Ul (pdu^j J T u - U2 J T U2 - u ^du^j du^j (x)du. 

h{T) = ^ J Tr- u y(^J T u -ui<fdui^ J Tu-u 2 (^J T U2 - u ^du^j du^j {x)du 



Note that in all the integrals Tr-u can be omitted since the counting measure is invariant for 
For (p = l{o}, using (13.51) and then the estimate (I3.8P we have 



C 
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Similarly, 
Notice that 

J2(GMy)) 2 = I I Ps(0,y)p t {0,y)dsdt 

= [ [ p s+ t(0,0)dsdt<CT 1 -~< (3.30) 
Jo Jo 

by ([33]) • Hence J 2 (T) -> 0. 

Also J 3 (T) ^ by (ESJ) and fl3T30|) . 

Finally, applying consecutively the Schwarz and Young inequalites (for the convolution on 
Qm) and f )3.30p we have 

h(T) < |a E GMx)G T (G TV ) 2 (x) (3.31) 



T 



E^^WE G T(G T ^) 2 (a;) 



j>(3- 7 )/2 

< C ^-J E (^iwW) 2 E 



< CiT- 7/2 0. 

Hence f l3T28|) follows by fl3T29|) . 

It remains to prove tightness. From fl3.23p - fl3.25p . for s < t we obtain 

E((X T (t),<f) - (X T (s), V )) 2 = J X {T) + J 2 (T) (3.32) 

where 

Ji{T) = T 7 I I p Tlv . ul {0,0)dudv, (3.33) 

J S J S 

/ / p T (u+v-2r){0,0)drdudv. (3.34) 

J s Jo 



We have 

-t 



rt—s 

Ji(T) < 2T 7 (t-s) / p T „(0,0)cfo 

Jo 

/ f T(t-s) \ 1-7 / ,T(t~a) 

= 2T^-\t-s)\J p v (0,0)dvl ly p v (0,0)dt; 



< C(£-s) 2 ~ 7 , (3.35) 
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by d3HD and since p v (0, 0) < 1. 
Next, by (EHJ), 



/ / \u + v -2r\^'- l drdudv (3.36) 

J s JO 



t r t 



< Ct \u-v\-^dudv = C 2 {t-s) 2 -\ (3.37) 

J s J s 

Combining fl3.32p -( l3~3Tl) we obtain tightness. □ 
Proof of Theorem 2.6 

The proof is a modification of the proof of Theorem 2.4. Again, we have (I3.23I) - (I3.25I) since 
Ht cancels out (see (12.171) ). The limit of covariances is obtained in the form (12.91) by proving 
( 13.26P and (13.271) . The only difference is that in (13.261) we use ( 13.61) instead of ( 13.81) . To prove 
convergence of finite dimensional distributions we show ( 13.28!) . St has the form ( 13.291) with 
HxIiiT) instead of Ii(T),i = 1, ... ,4. Convergence to of HTli(T),i < 3, is easily obtained 
from (13.61) and (I3.30p without any condition on the rate of divergence of Ht- For HtI^T) we 
use (13.311) . hence, by (13. 6p . 



H T h{T) < C — ^ / Pt{0,x)dt^2 / Ps{x,y)ds / p u {0,y)du 

T , c n..^0 „ ^0 JO 



T pT pT 



1 



»T pT pT 



JO JO 



< dT-" //2 H T 1/2 ->• 0, 

by (I2.16p . where in the last estimate we have used (13 .4p . Hence ( I3.28P follows. 

To prove tightness on [0,r] for any fixed r we go to ( 13.321) - (13.341) (Ht cancels out). As 
7 < 0, we have 

Ji(T) <C(t-s) 2 , 

and from (I3.36j) it follows that 

t pt 

2 



MT) <C J J („ + vY^dudv < C(r)(t - s) 2 . 

□ 

Proof of Proposition 2.8 

Modifying the argument at the beginning of the proof of Theorem 2.2 we have for (p, ip 6 Bf, 
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and s < t, 



Cov((X T (s), V ),(X T (t),^)) = Ee^ I I v(x)T T \u- v Mx)dudv 

T iefi A / 

tt-s rT{t~v) ps nT{s-u) 

I I ip{x)T u ip{x)dudv + / / ip(x)%ip(x)dvdu 
Jo Jo Jo Jo 

2E6s <p(x)Ghp(i 



as T — » oo. 

To prove the convergence in law of the sum J^fcLi(Xr(^fc)j yjfc), v?i, . . . , (fk E £>&, £1, . . . , ifc > 
0, we use the Lyapunov criterion, which again reduces to showing (13.21j) with T instead of T n 
and 9? > 0. Formula (13.221) implies that 



T 1 



Proof of Proposition 2.10 

(a) follows from (12.31) and the known formulas 

2 f°°sm 2 {tu) 



□ 



~ Jo u 2 



du, t > 0, (3.38) 



2 f°° sin 2 (t«) 



1 _ e - rt = -r / \ ' du, t > 0. (3.39) 

7r ,/ H/4 + u 

(b) is a direct consequence of (a) and (II. 3p . 

(c) Formula (I2.23P for osfBm also follows from (a) and (II. 4p . To derive it for onsfBm we 
interchange the sum and the integrals in (I2.9p . and after integrating we use (I3.38P and (13. 39 p . 

Comments on the proofs of Propositions 2.12, 2.14, 2.16, 2.17 and 2.20 

In Proposition 2.12, f l2.24[) -( l2.26j) follow by direct calculations (for osfBm and onsfBm ( 12. 9 p is 
again useful here). Formula ( I2.24p has already appeared in (I3.18p . 

Proposition 2.14 can also be verified directly (convergence in L 2 and hence almost surely 
as well, by independence). 

In Proposition 2.16 formula (I2.29P is also immediate once we know that the process g 

exists. hP t defined by (l2T3Tj) is finite for U E (0, 1/2) by (JS3}. Positive definiteness of (12T30]) 
follows from 

_L( e -6a^| s -t| _ e -W(»+t)) = 2 / l [0iS] (r)e- (s - r)6aJ l [0 , t ](r)e ( *- r)baJ cir. 

00? Jn 
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In Proposition 2.17 the estimates (I2.32j) for ofBm follow from ( 12. 4p . For osfBm and onsfBm 
we use (12. 9p and (13. 3p . and then f !2.32|) is a consequence of the corresponding estimates for 
sfflm and nsfflm (see (2.4) and (2.5) in jBGTOj and Proposition 3.4 |BGT5j l 

Finally, Proposition 2.20 can be also reduced to the corresponding properties of the non- 
oscillatory processes employing (I3.18p . ( 12. 9 p and (13. 3p . 

Proof of Proposition 2.18 

(a) First we consider ofBm $ = £ H . By stationarity of increments we have EV £ (C, H )/h £ = T, 
hence for the convergence in L 2 in (I2.34[) it suffices to prove that 



(3.40) 



limVar f ^-V £ (£ H ) ) = 0. 
e^o \h £ J 

Using (I2.33P and the identity 

EX 2 Y 2 = EX 2 EY 2 + 2(EXY) 2 , (3.41) 
which holds for any centered Gaussian vector (X,Y), we obtain 

= 2^ j\l £ (s,t)) 2 dsdt, (3.42) 
where 

Us,t) = -^j-E(& £ - ef)(^ e - ) > 0. (3.43) 

£ n £ 

By the Schwarz inequality, 

/ e (M)<l. (3.44) 
On the other hand, for s < t, from (I2.24[) and (13. 3[) we have 

I £ (s,t) < ^ J \u-v\ 2H ~ 2 dudv ^0 

since H e (1/2, 1). Hence (I3T401 follows. 

For the proof of almost sure convergence we need a finer estimate of the variance. It is easy 
to see that for t — s > 2e we have 

I £ (s,t)<Ce 2 - 2H \t-s\ 2H - 2 , 

hence, by (13.441) . 

(Is(s,t)) 2 < d (u-^— j-j for any < r < 4-4iJ. 



Therefore, from (13.421) we obtain 

Var (^V £ (Z H ^j < C 2 Te + C 3 (T)s r < C(T)s r 



(3.45) 



for any < r < (4 - AH) A 1. 
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Now we take e n and S as in the Proposition and fix 5 < r < (4 — AH) A 1. Using 

P (|^(^) " T | > 4T 5)/2 ) < e~ 5) Var (^(**)) , 

(I3.45P and the Borel-Cantelli lemma, the assertion follows. 

Next we prove (12.341) for osfBm $ = £ . Let be a continuous centered Gaussian process 
with covariance (12.101) independent of ( H . We know that ( H + r] H /a/2 is ofBm. Hence it is easy 
to see that by (12. 34ft for ofBm, the proof reduces to showing that 

\]mVJn H ) = 

in L 2 and almost surely. The L 2 convergence follows easily from (I3.4ip and the estimate 

rt+e rt+e 

\E(v? +e ~ vfM+e - V?)\ <C / (u + v) 2H - 2 dudv, 

Jt Js 

which is a consequence of ( 13. 1 9[) and (13.31) . Recall that the expression in (I3.19P is, up to a 

constant, 

E(v?r]?l ff=(l-7)/2. 

In the proof of almost sure convergence we use the Holder continuity of the paths of r] H 
with any exponent q < H, and the representation 



Vt 1 = / r udu 
Jo 



for a Gaussian process r u (see Subsection 2.3.5). We then have 

»T rt+e 



V £ (t] H ) < Ce"- 2H [ [ £ \r u \dudt 
Jo Jt 



for q sufficiently close to H. 

(b) The proof is similar to that of part (a), and we skip it. 



□ 



Proof of Theorem 2.21 

We have 

G(p(x)= G (. x -y)v{y), 

where 

G ^ = { V 1 if M = i > i, (3 ' 46) 

with A — (M — c)/(Mb(c - 1)) and D given by (12371) (see |DGWlj . it can also be easily 
obtained from ( 13. 2ft ). 
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By (I2~T9D . for s < t, 

= 2(Gl Bt , l Bs ) = 2(1 + J), (3.47) 

where 

x£B B y£B B 

i = E E G (*-^)- ( 3 - 49 ) 

xG-B s y€B t \B s 

From the hierarchical structure of it follows that if x G -B s , then y £ B s if and 
only if x — y G Let 5j = {x G fi^f : N — The cardinalities of Si and are 

M i-1 (M - 1), z = 1, 2, . . . , and M\ % = 0, 1, . . ., respectively. Hence 



1 = EE G (^ MS EE G ^) 

x£B s y&B s i=0 y&Si 

= M s i^ + ^{M-l)M i - l ^^=DM s (^^j , (3.50) 

by flUS}. 

By the hierarchical structure of Qm, if M = s + /c and |x| < s, then |x — y\ = s + k. 
Therefore, for t > s, 

1 = E E E - y) 

fe=l yes s+k xeB s 

t—s , 

= E( M - 1 ) A//s+fe_lM " 



t 



c J \ c J 

Then fl3T47l) -( l3~511 imply ( 12391) . □ 

4 Comments 

In this section we discuss some possible extensions of the results of Section 2. 
4.1 Oscillatory processes with small parameters 

We have defined ofBm and osfBm for if G (1/2, 1) only, whereas fBm and sfBm are well defined 
for H G (0, 1). In the oscillatory case h given by ( 12.31) is infinite for H G (0, 1/2), so a direct 
extension is impossible. It seems that a natural way to define £ H and ( H for H G (0, 1/2) is 
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to consider the covariances (jl.3p and (jl.4p with ft, given by (12.3ip . These processes can be 
extended to the whole real line (see Remark 2.11(b)). We then have 




) 



t>o 



where g H is the process that appears in Proposition 2.16. In other words, J Q g^du is an onsfBm 
with parameter H + 1, so our definitions of £ H , ( H and g H yield the same relationships as in 
the non-oscillatory case (cf. |DZl IBGTOl IBGT5] ). Existence of £ H (positive definiteness of the 
covariance) follows from (13.391) . It is not clear what would be particle picture interpretations 
of the oscillatory processes £ H ,( H and q h with H G (0, 1/2), as the method used in [BT] for 
the non-oscillatory case is not applicable here. 

4.2 Models with immigration 

Similarly to the case of a-stable motions in R d , one can add immigration to the particle systems 
as in |GNR] . We assume that particles immigrate according to a space-time uniform Poisson 
random field on Qm x R+- The new particles evolve independently, undergoing c-random walks 
with or without branching. In [GNRj limits of covariances of occupation time fluctuations 
were calculated, thus some new covariance functions were obtained, and the corresponding 
Gaussian processes were long range dependent. Analogous calculations can be carried out for 
the hierarchical case. 

In the non-branching case, for 7 < (c < 1),T„ = a~ n and F n = Tl 7/2 , taking for 
simplicity <p = l{o} ; it can be shown that 



lim Cov{(X Tn {s),cp), (X Tn (t),<p)) = K uUt- u)^~h t _ u + (s - u)^fi s -u) du, (4.1) 



where h is given by (12.311) with U = —7/2. The right-hand side of (14.11) is an oscillatory 
analogue of the covariance of weighted fractional Brownian motion with parameters 1 and —7 
as defined in [BGT5j . Another form of the covariance (14. ip is 



where £ H is ofBm with H — (1 — 7)/2. 

In the branching case, for < 7 < 1 (1 < c < \[M\T n = a~ n ,F n = Tn "'^ 2 , l f = l{o}> it 
can be shown that the limit of covariances is, up to a constant, the difference of the covariance 
(14. ip with 7 — 1 instead of 7, and the covariance of onsfBm with H = (3 — 7)/2. 

We have not attempted to prove convergence of finite dimensional distributions of Xx n - 

4.3 Other initial configurations 

One can consider initial conditions different from those described in Subsection 2.1. For exam- 
ple, it is possible to study a branching system starting from an equilibrium measure which is 
known to exist for 7 > (see Appendix in [DGWlj . and |GW2] ). By analogy with the result 





•sAt 
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for the a-stable case in M. d obtained in |M1] . one should expect that in the hierarchical model 
with < 7 < 1 the limit process is ofBm. Preliminary calculations confirm this conjecture. 

Another possibility is to consider inhomogeneous initial configurations analogous to those 
studied in |BGT6] and |BGT8j . 

4.4 More general random walks 

It is possible to consider general r^-random walks as described in the Introduction, i.e. (r 3 -) j=i,2,... 
is a probability distribution not necessarily of the form (II. 71) . It seems that, similarly as in 
[BGT10J, a natural assumption in this case is the existence of a positive limit 

hm — — = a. 

We conjecture that if this limit is strictly less than 1, then it plays the role of a (see (12. 2ft ) and 
the results should be similar to those obtained for c-random walks. If a = 1, then the limits 
should be as in Proposition 2.8 or Remark 2.9(a). The degree is given by 7 = log]/ a M — 1 if 
a < 1, and 7 = 00 if a = 1 (see the Appendix of [BGT10] . or |DGW4j l The case of "small" 7 
seems more difficult. 

4.5 Infinite variance branching 

Analogously as in [BGT3J IBGT4] , instead of binary branching one can consider a branching 
mechanism with probability generating function 

(1 - s) 1+ls 

S + L 7T7^ 0<s<1 ' 

where < (3 < 1. The assumption < 7 < 1 in Theorem 2.4 is replaced by 1/(3 — 1 <7 < 1/(3, 
and Ft = T^ 2 ~^^ 1+ ^ . Some preliminary calculations yield (1 + /3)-stable limits of rather 
complicated forms. Again they have dependent increments. 

Note that the condition 1/(3 — 1 < 7 implies persistence of the system, since it can be shown 
that 

/■oo 

/ (T t <p(x)) 1+ Pdt < 00, cpeB b , <p>0, 

which is sufficient for persistence (see the Appendix of |DGW1] ). Persistence means that the 
empirical process N t has a limit in distribution as t — > 00, N^, with full intensity measure, 
i.e., ENoc = EN , assuming that iVo is homogeneous Poisson (see |GWlj IGW2] and references 
therein) . 

4.6 Oscillatory bi-fractional Brownian notion 

Bi-fractional Brownian motion has covariance function 
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with parameters H E (0, 1], K E (0, 1]. It is an extension of fBm which was obtained analytically 
[HV]. In the same way as in |HVj . one can obtain an oscillatory analogue of offim with 
covariance function 




h defined by f l2.3|) . He (1/2, 1], K E (0, 1). We have not found particle picture interpretations 
for bi-fractional Brownian motion and its oscillatory analogue. 
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